Virtual provisioning with implementation resource boundary awareness

ABSTRACT

Virtual resources may be provisioned in a manner that is aware of, and respects, underlying implementation resource boundaries. A customer of the virtual resource provider may specify that particular virtual resources are to be implemented with implementation resources that are dedicated to the customer. Dedicating an implementation resource to a particular customer of a virtual resource provider may establish one or more information barriers between the particular customer and other customers of the virtual resource provider. Implementation resources may require transition procedures, including custom transition procedures, to enter and exit dedicated implementation resource pools. Costs corresponding to active and inactive implementation resources in a dedicated pools associated with a particular customer may be accounted for, and presented to, the customer in a variety of ways including explicit, adjusted per customer and adjusted per type of virtual resource and/or implementation resource.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.12/894,496, filed Sep. 30, 2010, entitled “VIRTUAL PROVISIONING WITHIMPLEMENTATION RESOURCE BOUNDARY AWARENESS,” the contents of which isherein incorporated by reference in its entirety.

BACKGROUND

It has become common for individuals and organizations of various kindsto use computers to perform and/or assist with a wide variety of tasks.Rather than purchasing and maintaining physical computers, it isbecoming more and more common to provision virtual computer systems, andother virtual computing resources of various kinds, with a specializedprovider of such virtual resources. From a point of view of a customerof a virtual resource provider, the use of virtual computing resourcescan have a variety of advantages such as cost-efficiency and timelyresponse to changing computing needs. However, conventional virtualresource providers have various shortcomings.

Virtual resource providers may manage large fleets of physical computersincluding relatively high capacity computers each capable of hostingmultiple virtual computer systems. Virtual resource providers can use avariety of methods for assigning virtual computer systems to physicalhost computers. At some conventional virtual resource providers, aparticular virtual computer system provisioned for one customer mayshare a high capacity computer with virtual computer systems associatedwith multiple other customers. Such sharing may be unacceptable to oneor more of the customers for a variety of reasons including regulatoryrequirements, organizational policies and/or perceived data securityrisk. Some conventional virtual resource providers attempt to preventunacceptable sharing with methods that are detrimental to virtualresource provider effectiveness, to efficiency (includingcost-efficiency) and/or to other virtual resource provider advantages,from a customer point of view and/or from a provider point of view.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments in accordance with the present disclosure will bedescribed with reference to the drawings, in which:

FIG. 1 is a schematic diagram illustrating an example environment forimplementing aspects in accordance with at least one embodiment;

FIG. 2 is a schematic diagram depicting aspects of an example virtualresource provisioning architecture in accordance with at least oneembodiment;

FIG. 3 is a geographic map depicting aspects of an example geographicregion in accordance with at least one embodiment;

FIG. 4 is a schematic diagram depicting aspects of example data centersin accordance with at least one embodiment;

FIG. 5 is a schematic diagram depicting further aspects of the exampledata centers in accordance with at least one embodiment;

FIG. 6 is a schematic diagram depicting aspects of an example virtualcomputing system server in accordance with at least one embodiment;

FIG. 7 is a schematic diagram depicting aspects of exampleimplementation resource pools in accordance with at least oneembodiment;

FIG. 8 is a timing diagram depicting aspects of example dedicatedimplementation resource pool transition procedure timing in accordancewith at least one embodiment;

FIG. 9 is a graph depicting aspects of an example implementationresource utilization accounting scheme in accordance with at least oneembodiment;

FIG. 10 is a schematic diagram depicting aspects of an example virtualresource provider control plane in accordance with at least oneembodiment;

FIG. 11 is a flowchart depicting example steps for virtual provisioningin accordance with at least one embodiment;

FIG. 12 is a flowchart depicting example steps for determiningimplementation resource availability in accordance with at least oneembodiment;

FIG. 13 is a flowchart depicting example steps for allocating and/orattempting to allocate an implementation resource in accordance with atleast one embodiment;

FIG. 14 is a flowchart depicting example steps for establishing adedicated implementation resource pool in accordance with at least oneembodiment;

FIG. 15 is a flowchart depicting example steps for maintaining aninactive dedicated implementation resource buffer in accordance with atleast one embodiment;

FIG. 16 is a flowchart depicting example steps for tracking costs inaccordance with at least one embodiment; and

FIG. 17 is a flowchart depicting example steps for workflow inaccordance with at least one embodiment.

Same numbers are used throughout the disclosure and figures to referencelike components and features, but such repetition of number is forpurposes of simplicity of explanation and understanding, and should notbe viewed as a limitation on the various embodiments.

DETAILED DESCRIPTION

In the following description, various embodiments will be described. Forpurposes of explanation, specific configurations and details are setforth in order to provide a thorough understanding of the embodiments.However, it will also be apparent to one skilled in the art that theembodiments may be practiced without the specific details. Furthermore,well-known features may be omitted or simplified in order not to obscurethe embodiment being described.

In at least one embodiment, virtual resources may be provisioned in amanner that is aware of, and respects, underlying implementationresource boundaries. A virtual resource provider may provision each of avariety of types of virtual resource (e.g., virtual computer systems,virtual data stores, virtual network connections) with a particular setof implementation resources (e.g., data center space, physical servercomputers, networking hardware). At least some of the implementationresources may be capable of participating in the implementation ofmultiple virtual resource instances, each potentially associated with adifferent customer of the virtual resource provider. In at least oneembodiment, a set of customers (i.e., one or more customers) of thevirtual resource provider may specify that particular virtual resourcesare to be implemented with implementation resources that are dedicatedto the set of customers (i.e., implemented with “dedicatedimplementation resources”), and/or that particular implementationresources utilized to implement virtual resources of the set ofcustomers are to be dedicated implementation resources.

Implementation resource boundaries (e.g., physical separationboundaries) may correspond to information barriers (e.g., barriers tounintended information transfer and/or unauthorized data modification).Accordingly, dedicating an implementation resource to a particular setof customers of a virtual resource provider (thereby excluding othercustomers from the implementation resource) may establish one or moreinformation barriers between the particular set of customers and othercustomers of the virtual resource provider (i.e., may place theparticular set of customers “behind” the one or more informationbarriers). Alternatively, or in addition, dedication of implementationresources can improve virtual resource performance, for example, atleast in part by reducing unanticipated implementation resourcecontention. As a further alternative, or further in addition, dedicationof implementation resources can improve virtual resource faulttolerance, for example, by isolating virtual resources from operationfaults that occur at other implementation resources.

The virtual resource provider may maintain a general implementationresource pool containing implementation resources from which no customerof the virtual resource provider is excluded. The virtual resourceprovider may furthermore maintain one or more dedicated implementationresource pools containing implementation resources dedicated to one ormore sets of customers of the virtual resource provider. Each dedicatedimplementation resource pool may have an associated set of entry and/orexit procedures (collectively “transition procedures”) such asimplementation resource configuration including access configuration,storage medium formatting, and secure data erase. Implementationresources in a dedicated pool may be active or inactive (i.e., activelyparticipating in implementing one or more virtual resources or idle).The virtual resource provider may monitor activity levels in dedicatedimplementation resource pools and transition implementation resourcesfrom and to the generation implementation resource pool to maintaintarget inactive to active implementation resource ratios and/or inactiveimplementation resource “buffers” with sizes based at least in part onforecast changes (e.g., rates of change) in activity levels.

There may be various costs, including financial costs, associated withproviding virtual resources to customers, and such costs may bepresented to customers of the virtual resource provider. Costs may bepresented in terms of virtual resource availability per unit time (e.g.,a monthly or hourly cost for each virtual computer system provisioned toa customer), in terms of units of data (e.g., gigabytes) processed,stored and/or transferred, and/or in terms of units of implementationresource actively utilized to implement virtual resources and/or madeunavailable to others. Costs corresponding to active and inactiveimplementation resources in a particular dedicated pool associated witha particular set of customers may be presented to those customers asseparate active and inactive amounts in a cost statement. Alternatively,a single amount may be presented to the customers based at least in parton the costs corresponding to maintaining the active and inactiveimplementation resources in the particular dedicated pool. As anotheralternative, a single amount may be presented to the customers based atleast in part on the costs corresponding to maintaining the active andinactive implementation resources for the virtual resource provider as awhole. In at least one embodiment, customers of the virtual resourceprovider may mitigate costs associated with implementation resources indedicated pools by reserving virtual resource instances and/orassociated dedicated implementation resources.

Various approaches may be implemented in various environments forvarious applications. For example, FIG. 1 illustrates aspects of anexample environment 100 for implementing aspects in accordance withvarious embodiments. As will be appreciated, although a Web-basedenvironment may be utilized for purposes of explanation, differentenvironments may be utilized, as appropriate, to implement variousembodiments. The environment 100 shown includes both a testing or adevelopment portion (or side) and a production portion. The productionportion includes an electronic client device 102, which may include anyappropriate device operable to send and receive requests, messages, orinformation over an appropriate network 104 and convey information backto a user of the device 102. Examples of such client devices includepersonal computers, cell phones, handheld messaging devices, laptopcomputers, tablet computers, set-top boxes, personal data assistants,electronic book readers, and the like.

The network 104 may include any appropriate network, including anintranet, the Internet, a cellular network, a local area network, a widearea network, a wireless data network, or any other such network orcombination thereof. Components utilized for such a system may depend atleast in part upon the type of network and/or environment selected.Protocols and components for communicating via such a network are wellknown and will not be discussed herein in detail. Communication over thenetwork may be enabled by wired or wireless connections, andcombinations thereof. In this example, the network 104 includes theInternet, as the environment includes a Web server 106 for receivingrequests and serving content in response thereto, although for othernetworks an alternative device serving a similar purpose could beutilized as would be apparent to one of ordinary skill in the art.

The illustrative environment 100 includes at least one applicationserver 108 and a data store 110. It should be understood that there maybe several application servers, layers, or other elements, processes, orcomponents, which may be chained or otherwise configured, which mayinteract to perform tasks such as obtaining data from an appropriatedata store. As used herein the term “data store” refers to any device orcombination of devices capable of storing, accessing, and/or retrievingdata, which may include any combination and number of data servers,databases, data storage devices, and data storage media, in anystandard, distributed, or clustered environment.

The application server 108 may include any appropriate hardware andsoftware for integrating with the data store as needed to executeaspects of one or more applications for the client device 102, and mayeven handle a majority of the data access and business logic for anapplication. The application server 108 provides access control servicesin cooperation with the data store 110, and is able to generate contentsuch as text, graphics, audio, and/or video to be transferred to theuser, which may be served to the user by the Web server 106 in the formof HTML, XML, or another appropriate structured language in thisexample.

The handling of all requests and responses, as well as the delivery ofcontent between the client device 102 and the application server 108,may be handled by the Web server 106. It should be understood that theWeb and application servers 106, 108 are not required and are merelyexample components, as structured code discussed herein may be executedon any appropriate device or host machine as discussed elsewhere herein.Further, the environment 100 may be architected in such a way that atest automation framework may be provided as a service to which a useror application may subscribe. A test automation framework may beprovided as an implementation of any of the various testing patternsdiscussed herein, although various other implementations may be utilizedas well, as discussed or suggested herein.

The environment 100 may also include a development and/or testing side,which includes a user device 118 allowing a user such as a developer,data administrator, or tester to access the system. The user device 118may be any appropriate device or machine, such as is described abovewith respect to the client device 102. The environment 100 may alsoinclude a development server 120, which functions similar to theapplication server 108 but typically runs code during development andtesting before the code is deployed and executed on the production sideand becomes accessible to outside users, for example. In someembodiments, an application server may function as a development server,and separate production and testing storage may not be utilized.

The data store 110 may include several separate data tables, databases,or other data storage mechanisms and media for storing data relating toa particular aspect. For example, the data store 110 illustratedincludes mechanisms for storing production data 112 and user information116, which may be utilized to serve content for the production side. Thedata store 110 also is shown to include a mechanism for storing testingdata 114, which may be utilized with the user information for thetesting side. It should be understood that there may be many otheraspects that are stored in the data store 110, such as for page imageinformation and access right information, which may be stored in any ofthe above listed mechanisms as appropriate or in additional mechanismsin the data store 110.

The data store 110 is operable, through logic associated therewith, toreceive instructions from the application server 108 or developmentserver 120, and obtain, update, or otherwise process data in responsethereto. In one example, a user might submit a search request for acertain type of item. In this case, the data store 110 might access theuser information 116 to verify the identity of the user, and may accessthe catalog detail information to obtain information about items of thattype. The information then may be returned to the user, such as in aresults listing on a Web page that the user is able to view via abrowser on the user device 102. Information for a particular item ofinterest may be viewed in a dedicated page or window of the browser.

Each server typically will include an operating system that providesexecutable program instructions for the general administration andoperation of that server, and typically will include a computer-readablemedium storing instructions that, when executed by a processor of theserver, allow the server to perform its intended functions. Suitableimplementations for the operating system and general functionality ofthe servers are known or commercially available, and are readilyimplemented by persons having ordinary skill in the art, particularly inlight of the disclosure herein.

The environment 100 in one embodiment is a distributed computingenvironment utilizing several computer systems and components that areinterconnected via communication links, using one or more computernetworks or direct connections. However, it will be appreciated by thoseof ordinary skill in the art that such a system could operate equallywell in a system having fewer or a greater number of components than areillustrated in FIG. 1. Thus, the depiction of the system 100 in FIG. 1should be taken as being illustrative in nature, and not limiting to thescope of the disclosure.

In at least one embodiment, one or more aspects of the environment 100may incorporate and/or be incorporated into a virtual resourceprovisioning architecture. FIG. 2 depicts aspects of an example virtualresource provisioning architecture 200 in accordance with at least oneembodiment. The example virtual resource provisioning architecture 200includes multiple clients 202-204 communicatively connected to a virtualresource provider 206 over a network 208. For example, the clients202-204 may corresponding to computing devices such as the computingdevice 102 of FIG. 1 and/or client programs incorporated into suchcomputing devices. The ellipsis between the client 202 and the client204 indicates that the virtual resource provisioning architecture 200may include any suitable number of clients (e.g., thousands, millions,and more) although, for clarity, only two are shown in

FIG. 2.

One or more of the clients 202-204 may be utilized by one or morecustomers of the virtual resource provider 206 to interact with acontrol plane 210 of the virtual resource provider 206, and therebyprovision one or more virtual resources 212. Alternatively, or inaddition, one or more of the clients 202-204 may be utilized (notnecessarily by virtual resource provider 206 customers) to interact withprovisioned virtual resources 212. The provisioned virtual resources 212may include any suitable virtual resources. Examples of suitable virtualresources 212 include virtual computer systems 214, virtual networkconnections 216, and virtual data stores 218, as well as virtualresources not shown in FIG. 2 such as specialized data processingagents, media streaming agents including audio and video streamingagents, message queues, publish-subscribe topics configured to notifysubscribers having subscriptions that match events published to thepublish-subscribe topics, monitoring agents, load balancing agents, andsuitable combinations thereof.

The virtual resource provider 206 may include any suitableimplementation resources 220. Each of the virtual resources 212 may beimplemented by a set of the implementation resources 220. In at leastone embodiment, various implementation resources of the implementationresources 220 may be configured to participate in implementing, at leastin part, multiple virtual resources of the virtual resources 212.Examples of suitable implementation resources 220 include virtualcomputer system (VCS) servers 222, network hardware 224, and data storeservers 226, as well as implementation resources not shown in FIG. 2and/or those described in more detail below with reference to FIG. 3,FIG. 4, FIG. 5 and FIG. 6. The control plane 210 may process virtualresource provisioning requests, manage allocation of virtual resources212 to implementation resources 220 and/or manage allocation ofimplementation resources 220 to virtual resources 212, as well asprovide for associated cost accounting services. An example virtualresource provider control plane in accordance with at least oneembodiment is described below in more detail with reference to FIG. 10.

When a particular implementation resource of the implementationresources 220 participates in the implementation of multiple virtualresources of the virtual resources 212, the implementation resource maybecome contended, for example, the implementation resource may receivesufficient service requests from the multiple virtual resources thatrequest servicing time increases. Contended implementation resources canbe a source of unintended and/or unauthorized information transferbetween virtual resources, for example, based at least in part onvariation in request servicing time. In at least one embodiment, a setof customers may establish a barrier to such information transfer toother customers of the virtual resource provider 206 at least in part byrequesting that virtual resources associated with the set of customersbe provisioned with dedicated implementation resources. Such barriersmay lower a probability that one or more of the other customers of thevirtual resource provider gains unauthorized read and/or write access toinformation (including unpublished information) concerning the virtualresources of the set of customers.

Implementation resource dedication boundaries, defining implementationresource dedication units, may correspond to boundaries between physicaland/or hardware components including boundaries due to physical barriersand/or physical separations, as well as to hard (e.g., hardwareenforced) scheduling and/or timing boundaries, and suitable combinationsthereof.

Implementation resource dedication boundaries may also correspond togeographic boundaries in at least some embodiments. FIG. 3 depictsaspects of an example geographic region 300 in accordance with at leastone embodiment. The geographic region 300 includes the contiguous UnitedStates and several data centers 302-318. In at least one embodiment, theimplementation resources 220 (FIG. 2) of the virtual resource provider206 include the data centers 302-318 and/or corresponding geographicregions. The physical separation of the data centers 302-318 correspondsto implementation resource dedication boundaries, and may define thecorresponding geographical regions. A particular set of customers of thevirtual resource provider 206 may specify that virtual resourcesassociated with the set of customers be provisioned with a set ofdedicated data centers and/or within a set of dedicated geographicalregions. Although not shown in FIG. 3, one or more such data centersand/or corresponding geographic regions may be located internationally.

Implementation resource dedication boundaries and/or units may alsocorrespond to portions of a data center in at least some embodiments.FIG. 4 depicts aspects of example data centers 402, 404 in accordancewith at least one embodiment. The example data center 402 includesmultiple server racks 406-408 containing server computers such as theVCS servers 222 and the data store servers 226 of FIG. 2, as well asmultiple network racks 410 containing network hardware such as thenetwork hardware 224 of FIG. 2. Each such server rack 406-408 and/ornetwork rack 410 may correspond to an implementation resource dedicationunit, so that a particular set of customers of the virtual resourceprovider 206 may specify that virtual resources of the set of customersare to be provisioned with a set of dedicated server racks, a set ofnetwork racks, and/or a suitable combination thereof.

The data center 402 may require one or more types of identificationand/or access device (e.g., photo ID card, biometric authentication,physical key, purpose-encoded RFID card) that is capable of beingreliably authenticated (e.g., cryptographically) in order to gainphysical access to the data center 402. Accordingly, the data center 402may be associated with a security level and/or a security zone (e.g.,with respect to a publicly accessible space). In addition, the datacenter 402 may incorporate one or more higher level and/or differentsecurity zones. The data center 402 may include multiple accesscontrolled areas 412-418, for example, corresponding to locked roomswithin the data center 402. For example, it may be that only selectedand/or specially vetted data center 402 staff have access to the accesscontrolled areas 412-418. Such selection and/or special vetting of datacenter 402 staff may be part of transition procedures for accesscontrolled areas 412-418 into dedicated implementation resource pools.

The data center 402 may further include multiple access monitored areas420-426 not necessarily further secured against physical access. Forexample, access monitored areas 420-426 may log physical entry and/orexit of data center 402 staff, and/or otherwise document staff presencein the access monitored areas 420-426 including with audio and/or videorecordings. Each security zone including each access controlled areas412-418 and each access monitored areas 420-426 may correspond to animplementation resource dedication unit, so that a particular set ofcustomers of the virtual resource provider 206 (FIG. 2) may specify thatvirtual resources of the set of customers are to be provisioned with aset of dedicated security zones.

The data center 404 of FIG. 4 may be mobile. For example, the datacenter 404 may be constructed in a shipping container, and made mobileby a variety of means including the illustrated semi-trailer truck. Themobile data center 404 may include server racks, network racks, accesscontrolled areas and/or access monitored areas corresponding to theserver racks 406-408, network racks 410, access controlled areas412-418, and/or access monitored areas 420-426 of the immobile datacenter 402. The mobile data center 404 may also include suitable mobilepower generation hardware, and suitable wireless and/or wireline highspeed digital data links including one or more Internet links.

FIG. 5 depicts further aspects of data centers in accordance with atleast one embodiment. A data center 502 may include multiple serverracks 504-506. The data center 502 is an example of the data centers402, 404 of FIG. 4. The ellipsis between the server rack 504 and theserver rack 506 indicates that the data center 502 may include anysuitable number of server racks although, for clarity, only two areshown in FIG. 5. Each server rack 504-506 may participate in maintainingservices such as electric power and data communications to multipleserver computers 508-514 and 516-522. Again, the ellipses indicate thatthe server racks 504-506 may include any suitable number of servercomputers. For example, the server computers 508-522 may include one ormore VCS servers 222

(FIG. 2) and/or one or more data store servers 226. Each server 508-522may correspond to an implementation resource dedication unit.

In FIG. 5, each server rack 504-506 is depicted as including a rackswitch 524-526. The rack switches 524 and 526 may be responsible forswitching packets of digital data to and from their respective sets ofserver computers 508-514 and 516-522. Each rack switch 524-526 maycorrespond to an implementation resource dedication unit. However, inthe case (depicted in FIG. 5) that the server rack 504-506 includes onerack switch 524-526, dedicating the rack switch 524-526 to a particularset of customers of the virtual resource provider 206 (FIG. 2) causesdedication of the respective server rack 504-506. This is an example ofdedication of a component (an implementation resource dedicationsub-unit) of an implementation resource dedication unit causingdedication of the containing dedication unit (“containing unitdedication”) in accordance with at least one embodiment. Implementationresources dedication units may indicate which (if any) of theirsub-units cause containing unit dedication.

The rack switches 524-526 may be communicatively linked to a data centerswitching fabric 528 and then to a set of edge routers 530 that connectsthe data center 502 to one or more other computer networks including theInternet. The switching fabric may include any suitable set ofnetworking components including multiple interconnected switches 532-538(for clarity, only four are shown in FIG. 5) of one or more switch typesarranged in one or more switching layers, as well as routers, gateways,bridges, hubs, repeaters, firewalls, computers, and suitablecombinations thereof. In at least one embodiment, the rack switches524-526 and the edge routers 530 are considered part of the switchingfabric 528. The rack switches 524-526, the edge routers 530, and thecomponents of the switching fabric 528 are examples of the networkhardware 224 of FIG. 2.

Portions of the switching fabric 528, sets of switching fabric 528networking components such as sets of the switches 532-538, and/or theedge routers 530 may correspond to implementation resource dedicationunits. Alternatively, or in addition, a particular set of customers ofthe virtual resource provider 206 (FIG. 2) may specify that virtualresources of the set of customers be provisioned with a set of dedicateddata paths and/or channels (collectively, “data paths”) through theswitching fabric 528 and/or the edge routers 530. With respect todedicated data paths, the implementation resource dedication units maycorrespond to physical data paths such as sets of wires and/or cables.Alternatively, or in addition, the implementation resource dedicationunits may correspond to hard scheduled communication time slots in asynchronous communication scheme.

Implementation resource dedication boundaries and/or units may alsocorrespond to portions of a server computer in at least someembodiments. FIG. 6 depicts aspects of an example VCS server 602 inaccordance with at least one embodiment. The VCS server 602 of FIG. 6 isan example of the VCS servers 222 of FIG. 2. Virtual resources 604implemented by the VCS server 602 may include multiple virtual computersystems (VCS) 606-614 of various types. The virtual resources 604 may beimplemented with any suitable implementation resources 616 of the VCSserver 602. Examples of suitable implementation resources 616 includeone or more processors 618 such as central processing units (CPUs) andmulti-core CPUs, one or more volatile storage devices 620 such asrandom-access memory (RAM), one or more non-volatile storage devices 622such as flash memory and hard disk drives (HDDs), and/or one or morenetwork interfaces 624 such as network interface cards (NICs). Eachprocessor 618, volatile storage device 620, non-volatile storage device622, and/or network interface 624 may correspond to an implementationresource dedication unit.

Each of the virtual computer systems 606-614 may be implemented with aset of the implementation resources 616. Different types of the virtualcomputer systems 606-614 may be implemented with different sets of theimplementation resources 616. For example, a “large” type virtualcomputer system may require more implementation resources than a “small”type virtual computer system. A “memory intensive” type of virtualcomputer system may require an additional portion of the volatilestorage device(s) 620 implementation resource. A “processing intensive”type of virtual computer may require an additional portion of theprocessor(s) 618 implementation resource. The example depicted in FIG. 6shows the VCS server 602 maintaining four “small” virtual computersystems 606-612 and one “large” virtual computer system 614. This is anexample of a virtual resource implementation capacity of the VCS server602. Of course, other configurations are possible. For example, the VCSserver 602 may be able to maintain eight “small” virtual computersystems absent the “large” virtual computer system 614, and so on.

The “small” type virtual computer system may correspond to a minimalvirtual resource maintained by the VCS server 602 and/or to a unitvirtual computer system cost. The virtual resource provider 206 (FIG. 2)may measure, estimate and/or determine costs for other types of virtualcomputer system, including virtual computer systems implemented withdedicated implementation resources, in terms of the number of “small”type virtual computer systems displaced by the type. For example, aparticular set of customers of the virtual resource provider may specifythat virtual computer systems of the set of customers are to beimplemented with dedicated hard disk drives. If the VCS server 602 istypically configured to share each hard disk drive with two “small”virtual computer systems, then the virtual computer systems withdedicated hard disk drives displace at least two such “small” virtualcomputer systems.

In at least one embodiment, dedicated implementation resources are drawn(i.e., allocated) from dedicated implementation resource pools. Forexample, a dedicated implementation resource pool may be establishedcorresponding to each set of customers of the virtual resource provider206 (FIG. 2) that requests the provisioning of virtual resources 212with dedicated implementation resources, and the dedicatedimplementation resources that implement the requested virtual resourcesmay then be drawn from the dedicated implementation resource poolassociated with the set of customers. FIG. 7 depicts aspects of exampleimplementation resource pools in accordance with at least oneembodiment.

FIG. 7 depicts a general implementation resource pool 702 and threededicated implementation resource pools 704, 706, 710.

In this example, implementation resources 220 (FIG. 2) begin in thegeneral implementation resource pool 702. The three dedicatedimplementation resource pools 704, 706, 710 may be established for threedifferent sets of customers of the virtual resource provider 206. Thededicated implementation resource pool 704 and the dedicatedimplementation resource pool 710 have an overlapping portion 712.Implementation resources in the overlapping portion 712 may be utilizedto implement virtual resources of the set of customers associated witheither the dedicated implementation resource pool 704 or the dedicatedimplementation resource pool 710. Alternatively, or in addition, theoverlapping portion 712 may be configured as a distinct dedicatedimplementation resource pool 712 associated with virtual resources,including joint virtual resources, of the joint set of customersassociated with both the dedicated implementation resource pool 704 andthe dedicated implementation resource pool 710.

Implementation resources 220 may be transitioned from the generalimplementation resource pool 702 to the dedicated implementationresource pools 704, 706, 710, 712 in accordance with implementationresource transition procedures specified by at least one of therespective set of customers, thereby dedicating the transitionedimplementation resources to the set of customers. Implementationresources 220 may be transitioned to the dedicated implementationresource pools 704, 706, 710, 712 responsive to virtual resourceprovisioning requests. Such requests may fail and/or be delayed withrespect to fulfillment when the general implementation resource pool 702contains insufficient implementation resources of the type(s) requiredto fulfill the requests. To avoid such problems, virtual resourceprovider 206 customers may establish reserved implementation resourcepools.

In the example depicted in FIG. 7, the set of customers associated withthe dedicated implementation resource pool 706 has established areserved implementation resource pool 708 (e.g., with the control plane210 of FIG. 2). In at least one embodiment, implementation resources 220in the reserved implementation resource pool 708 remain in the generalimplementation resource pool 702 until required to fulfill a virtualresource provisioning request. However, when required, implementationresources 220 in the reserved implementation resource pool 708 aretransitioned to the dedicated implementation resource pool 706 evenwhen, at the time, those implementation resources are participating inthe implementation of virtual resources of one or more other customers.That is, the set of customers associated with the reservedimplementation resource pool 708 is given priority to the implementationresources 220 in the reserved implementation resource pool 708. Thevirtual resources of preempted customers may be migrated toimplementation resources in the general implementation resource pool 702or, when the general implementation resource pool 702 does not containsufficient implementation resources of the appropriate type(s), thevirtual resources may be de-provisioned.

Although some dedicated implementation resource pool transitionprocedures can be fast (e.g., on the order of milliseconds), sometransition procedures, particularly custom transition proceduresspecified by at least one customer of the virtual resource provider 206,require a significant amount of time to complete (e.g., seconds,minutes, hours, and more). FIG. 8 depicts aspects of example dedicatedimplementation resource pool transition procedure timing in accordancewith at least one embodiment.

At a time to, an implementation resource 802 begins in a generalimplementation resource pool 804. For example, the implementationresource 802 may be one of the implementation resources 220 of FIG. 2,and the general implementation resource pool 804 may correspond to thegeneral implementation resource pool 702 of FIG. 7. At a time ti, avirtual resource provider 206 event occurs that requires a transition ofthe implementation resource 802 to a dedicated implementation resourcepool 806. For example, the event may be a virtual resource 212provisioning request, or an automated transition to meet forecastprovisioning requests, and the dedicated implementation resource pool806 may correspond to the dedicated implementation resource pool 704 ofFIG. 7.

At a time t₂, a dedicated implementation resource pool entry procedure808 may begin. The entry procedure 808 may be any suitableimplementation resource 802 (re)configuration procedure. Examples ofsuitable entry procedures include data storage media formatting, datastorage media erasure including secure data erasure procedures, softwareinstallation, component configuration, component reallocation, accesscontrol configuration, firewall reconfiguration, virtual resourcemigration (e.g., to alternate implementation resources), authenticationcredential creation and issuance, mobile data center 404 (FIG. 4)relocation, manual review of implementation resource transition bycustomer, customer-provided entry procedures including customer-providedcomputer-executable instructions and/or any suitable combinationthereof. In at least one embodiment, customers select the dedicatedimplementation resource pool entry procedure 808 from a set of suchprocedures offered by the virtual resource provider 206 (FIG. 2). Theset of offered procedures may be pre-defined and/or dynamicallygenerated by the virtual resource provider 206 for a particular set ofcustomers.

Until time t₃, when the entry procedure 808 is complete, theimplementation resource 802 may be in a transition state 810. While inthe transition state 810, the implementation resource 802 does notparticipate in implementing virtual resources 212 (FIG. 2). At time t₃,the entry procedure 808 completes 812, and the implementation resource802 enters the dedicated implementation resource pool 806. While in thededicated implementation resource pool 806, the implementation resource802 may participate in implementing virtual resources 212 of theassociated set of customers.

At a time t₄, another virtual resource provider 206 (FIG. 2) event mayoccur that requires a transition of the implementation resource 802 fromthe dedicated implementation resource pool 806 back to the generalimplementation resource pool 804. For example, the event may be avirtual resource 212 de-provisioning request, or an automated transitionto optimize a proportion of idle implementation resources 220 in thededicated implementation resource pool 806. At a time t₅, a dedicatedimplementation resource pool exit procedure 814 may begin. The exitprocedure 814 may be any suitable implementation resource 802(re)configuration procedure including those described above for theentry procedure 808 and their logical and/or procedural complements.Again, in at least one embodiment, customers select the dedicatedimplementation resource pool exit procedure 814 from a pre-defined setof such procedures.

Until time t₆, when the exit procedure 814 is complete, theimplementation resource 802 may again be in the transition state 810. Attime t₆, the exit procedure 814 completes 816, the implementationresource 802 exits the dedicated implementation resource pool 806 andreenters the general implementation resource pool 804. After time t₆,the implementation resource 802 may repeatedly enter and exit thededicated implementation resource pool 806, and/or other dedicatedimplementation resource pools not shown in FIG. 8, until theimplementation resource 802 reaches the end of its service life at timet₇.

As described above with respect to FIG. 6, some implementation resourcesmay implement multiple virtual resources (i.e., may be “apportionable”implementation resources). For example, the VCS server 602 may implementmultiple virtual computing systems. In such cases it can happen that oneportion of an apportionable implementation resource is allocated toimplementing one or more virtual resources (is “allocated”), whileanother portion remains unallocated. When the apportionableimplementation resource is dedicated to a particular set of customers ofthe virtual resource provider 206 (FIG. 2), the unallocated portion isfurthermore unavailable for allocation to other customers of the virtualresource provider, and this may correspond to a decreased probabilitythat the dedicated apportionable implementation resource will achieve100% utilization. From the point of view of the virtual resourceprovider 206, this is an opportunity cost. Such costs can be accountedfor in a variety of ways. FIG. 9 depicts aspects of an exampleimplementation resource utilization accounting scheme in accordance withat least one embodiment.

FIG. 9 depicts utilization of an example apportionable implementationresource incorporating four units of resource allocation (or “resourceallocation units”) each capable of independent participating in theimplementation of one or more virtual resources 212 (FIG. 2). Forexample, the example apportionable implementation resource may be one ofthe VCS servers 222 capable of implementing four “small” type virtualcomputer systems, or one “large” type virtual computer system. Thevirtual resource implementation capacity of an apportionableimplementation resource may be measured in resource allocation units.For example, the implementation capacity of the VCS servers 222 may bemeasured in terms of a standardized virtual computer system unit such asthe “small” type virtual computer system.

FIG. 9 further depicts utilization of the example apportionableimplementation resource over eight units of time (or “time units”, e.g.,seconds, hours, months, and so on). During the first two time units, oneof the four resource allocation units is allocated (as indicated by thesquare shaded with diagonal lines), and may be active (i.e., activelyparticipating in the implementation of at least one virtual resource),while three of the four resource allocation units are unallocated (asindicated by the unshaded squares), and are inactive. In at least oneembodiment, allocated resource allocation units that are idle are alsoconsidered inactive. During the third time unit, each of the fourresource allocation units are allocated. During the fourth time unit,two of the four resource allocation units are allocated. For theremaining four time units, one of the four resource allocation units areallocated.

Each square in FIG. 9, shaded or unshaded, corresponds to a resourceutilization amount, namely a multiplicative product of a resourceallocation unit and a time unit (herein called a “resource hour” forclarity). For example, VCS server utilization may be measured in “small”VCS hours. If the example apportionable implementation resource were inthe general implementation resource pool 702 (FIG. 7), 4×8=32 resourcehours would be available for allocation. In the example utilizationdepicted in FIG. 9, 12 resource hours are actually allocated over the 8“hour” time period, resulting an opportunity cost of 20 resource hoursto the virtual resource provider 206 (FIG. 2) if the exampleapportionable implementation resource is dedicated to a particularcustomer.

Provisioning, configuration, re-configuration, and/or de-provisioning(collectively, “provisioning”) of virtual resources may be controlled bythe control plane 210 (FIG. 2) of the virtual resource provider 206.FIG. 10 depicts aspects of an example control plane 1002 in accordancewith at least one embodiment. The control plane 1002 of FIG. 10 is anexample of the control plane 210 of FIG. 2. The control plane 1002 mayinclude a provisioning interface 1004 configured at least to receivevirtual resource 212 provisioning requests from one or more of theclients 202-204, a workflow component 1006 configured at least to guideresponses to provisioning requests in accordance with one or moreprovisioning workflows, a resource allocation component 1008 configuredat least to manage allocation of implementation resources 220 to virtualresources 212, and an accounting component 1010 configured at least totrack and present costs associated with the virtual resources 212 and/orthe implementation resources 220.

The provisioning interface 1004 may include any suitable provisioninginterface elements. Examples of suitable provisioning interface elementsinclude interface elements that correspond to requests to provision,configure, reconfigured and/or de-provision the virtual resources 212(FIG. 2), as well as interface elements that provide access to virtualresource 212 configuration information, and one or more interfaceelements enabling authentication to establish authority for suchprovisioning-related operations. The provisioning interface 1004 mayincorporate and/or be incorporated in a user interface (UI) such as agraphical user interface (GUI), a Web-based interface, a programmaticinterface such as an application programming interface (API) and/or aset of remote procedure calls (RPCs) corresponding to provisioninginterface elements, a messaging interface such as a messaging interfacein which the interface elements of the provisioning interface 1004correspond to messages of a communication protocol, and/or any suitablecombination thereof.

In at least one embodiment, the provisioning interface 1004, theresource allocation component 1008, and the accounting component 1010may create, and/or cause the workflow component 1006 to create, one ormore workflows that are then maintained by the workflow component 1006.Workflows, such as provisioning workflows and policy enforcementworkflows, may include one or more sequences of tasks to be executed toperform a job, such as provisioning or policy enforcement. A workflow,as the term is used herein, is not the tasks themselves, but a taskcontrol structure that may control flow of information to and fromtasks, as well as the order of execution of the tasks it controls. Forexample, a workflow may be considered a state machine that can manageand return the state of a process at any time during execution.Workflows may be created from workflow templates. For example, aprovisioning workflow may be created from a provisioning workflowtemplate configured with parameters by the resource allocation component1008. As another example, a policy enforcement workflow may be createdfrom a policy enforcement workflow template configured with parametersby the resource allocation component 1008.

The workflow component 1006 may modify, further specify and/or furtherconfigure established workflows. For example, the workflow component1006 may select particular implementation resources 220 (FIG. 2) of thevirtual resource provider 206 to execute and/or be assigned toparticular tasks. Such selection may be based at least in part on thecomputing resource needs of the particular task as assessed by theworkflow component 1006. As another example, the workflow component 1006may add additional and/or duplicate tasks to an established workflowand/or reconfigure information flow between tasks in the establishedworkflow. Such modification of established workflows may be based atleast in part on an execution efficiency analysis by the workflowcomponent 1006. For example, some tasks may be efficiently performed inparallel, while other tasks depend on the successful completion ofprevious tasks.

As part of provisioning a virtual resource, the provisioning interface1004 and/or the workflow component 1006 may request that the resourceallocation component 1008 determine the appropriate set of theimplementation resources 220 (FIG. 2) required to implement the virtualresource, determine whether the required implementation resources areavailable and/or in accordance with virtual resource provider 206allocation policies, and/or allocate the required implementationresources. The resource allocation component 1008 may incorporate anysuitable resource allocation and/or resource scheduling algorithm. Suchalgorithms are well known to those of skill in art, and need not bedescribed here in detail. The resource allocation component 1008 mayinclude a dedicated resource pools component 1012 configured at least tomanage the establishment and maintenance of dedicated implementationresource pools such as the dedicated implementation resource pools 704,706 of FIG. 7.

The dedicated resource pools component 1012 may include a businesspolicy enforcement component 1014 configured at least to analyze anallocation request with respect to a set of resource allocation businesspolicies and provide an evaluation (e.g., permit or deny) with respectto whether the allocation request is in accordance with the set ofresource allocation business policies. Alternatively, or in addition,the business policy enforcement component 1014 may participate inallocation of implementation resources 220 (FIG. 2) to virtual resources212, and/or virtual resources 212 to implementation resources 220, toenforce the set of resource allocation business policies. The set ofresource allocation business policies may include any suitable resourceallocation policies. Examples of suitable resource allocation policiesinclude policies having conditions based on parameters such as typeand/or number of dedicated virtual resources requested, correspondingtype and/or number of dedicated implementation resources, cost plan ofrequesting customer, current levels and/or proportions of unallocatedimplementation resources, forecast levels and/or proportions of inactivededicated implementation resources (e.g., in the requesting customer'sdedicated implementation resource pool), and suitable combinationsthereof. Resource allocation policies may include any suitableconditions such as compound conditions specified with Boolean operatorsand conditions specifying that particular numbers, levels and/orproportions are above a minimum value, below a maximum value and/orwithin a specified range of values.

With respect to implementation resources that implement multiple virtualresources, virtual resource density or virtual resource spreadcorresponds to a number of virtual resources per unit implementationresource (e.g., the number of virtual computing systems per VCS server602). Customers of the virtual resource provider 206 (FIG. 2) mayrequest that provisioned virtual resources have a specified density orspread, for example, to reduce a probability that multiple of thecustomer's virtual resources will be affected by an implementationresource failure. High spreads (low densities) for virtual resourcesimplemented with dedicated implementation resources can result in poorimplementation resource utilization efficiencies (e.g., relatively highinactive implementation resource utilization to active implementationresource utilization ratios such as utilization ratios over 20%). In atleast one embodiment, the set of resource allocation business policiesmay include one or more policies having conditions based on virtualresource density and/or spread levels (e.g., setting one or more densityand/or spread limits). The dedicated resource pools component 1012 mayfurther include a spread component 1016 configured at least to allocatededicated implementation resources in accordance with permitted densityand/or spread levels. Alternatively, or in addition, the spreadcomponent 1016 may be configured at least to allocate and/or reallocatededicated implementation resources to achieve a maximum permitted and/orspecified virtual resource density, and/or to maximize a number of free(i.e., 0% allocated) implementation resource dedication units.

The dedicated resource pools component 1012 may further include a pooltransition component 1018 configured at least to transition and/ormanage the transition of implementation resources 220 to and fromdedicated implementation resource pools 704, 706 (FIG. 7). Theimplementation resources 220 (FIG. 2) may be associated with a set ofdedicated pool transition procedures. For example, each type ofimplementation resource may have an associated transition procedure.Alternatively, or in addition, customers of the virtual resourceprovider 206 may specify dedicated pool transition procedures forparticular types of virtual resource and/or implementation resource. Thepool transition component 1018 may identify a set of appropriatetransition procedures associated with a particular set of virtualresources to be provisioned with implementation resources that includeone or more dedicated implementation resources. The pool transitioncomponent 1018 may execute such transition procedures. Alternatively, orin addition, the pool transition component 1018 may create (or cause theworkflow component 1006 to create) one or more dedicated pool transitionworkflows corresponding to the identified set of dedicated pooltransition procedures.

The dedicated resource pools component 1012 may further include a poolutilization component 1020 configured at least to monitor activityand/or inactivity levels in dedicated implementation resource pools704-706 (FIG. 7), forecast activity and/or inactivity levels indedicated implementation resource pools 704-706, and initiateimplementation resource transitions to and from dedicated implementationresource pools 704-706. Implementation resources in dedicatedimplementation resource pools 704-706 may be inactive for a variety ofreasons. For example, implementation resources may have beentransitioned to the dedicated pools 704-706 in anticipation of virtualresource provisioning requests that have not yet occurred, or toparticipate in the implementation of virtual resources that have beende-provisioned. Such inactive implementation resources may be anunnecessary expense for the responsible customer and/or an opportunitycost for the virtual resource provider 206 (FIG. 2).

However, since the time to transition implementation resources from thegeneral implementation resource pool 702 (FIG. 7) may be significant,some level of inactivity in dedicated implementation resource pools704-706 (i.e., an inactive dedicated implementation resource “buffer”)may be desirable. The responsible customer and/or a virtual resourceprovider 206 (FIG. 2) administrator may specify one or more targetlevels of activity and/or inactivity for each dedicated implementationresource pool 704-706. In at least one embodiment, an activity and/orinactivity target may be set for each type of implementation resource.Activity and/or inactivity targets may be specified in terms of resourceallocation unit numbers, implementation resource dedication unitnumbers, proportions thereof, and/or derivatives thereof including ratesof change, change velocities, change accelerations, first derivatives,second derivatives, third derivatives, and any suitable levelderivative. The pool utilization component 1020 may forecast valuescorresponding to activity and/or inactivity targets based at least inpart on monitored activity and/or inactivity levels, and scheduleimplementation resource transitions to and from the dedicatedimplementation resource pools 704-706 to meet corresponding activityand/or inactivity targets based at least in part on the forecast values.

The accounting component 1010 may maintain an account for each customerof the virtual resource provider 206 (FIG. 2). Utilization and/or costsassociated with virtual resources 212 provisioned by a customer, and/orassociated implementation resources 220, may be recorded in thecustomer's account. The accounting component 1010 may maintain one ormore cost plans 1022 specifying how the costs are allocated to thecustomer's account. The accounting component 1010 may include an accountuser interface (UI) component 1024 configured at least to provide thecustomer with one or more presentations of the utilization and/or costsrecorded in the customer's account and, when one or more of the costscorrespond to a financial balance owed to the virtual resource provider206, one or more mechanisms for settling the account balance (e.g.,payment instrument processing). The account UI 1024 may further providefor account creation, account configuration and reconfiguration, accountdetails viewing and updating, as well as account deletion. Account(re)configuration may include selection from a qualified list of costplans 1022 when multiple such cost plans 1022 are available.

The cost plan(s) 1022 may specify a cost per resource hour correspondingto each type of virtual resource 212 and/or implementation resource 220(FIG. 2). Costs may be incurred by a customer for allocated and/oractive resource hours (“activity costs”), for example, at an activeimplementation resource cost rate. In at least one embodiment, costs mayalso be incurred by the customer for unallocated and/or inactiveresource hours associated with implementation resources in pools 704-706(FIG. 7) dedicated to the customer (“dedicated resource costs” or“inactivity costs”), for example, at an inactive implementation resourcecost rate. In at least one embodiment, activity costs and dedicatedresource costs are maintained separately in a customer's account, andexplicitly presented as separate items in corresponding cost statements.Alternatively, activity costs and/or rates may be adjusted to accountfor dedicated resource costs so that cost statements need not include anitem that explicitly states dedicated resource costs. The cost plan(s)1022 may include any suitable function of allocated, unallocated, activeand/or inactive resource hours including suitable linear functions andsuitable non-linear functions.

For example, activity costs and/or rates may be adjusted on a percustomer basis.

Cost statements may be generated periodically and each state costsincurred during a time period (the “cost statement time period”) elapsedsince a previously generated cost statement. Activity costs anddedicated resource costs incurred by a particular customer may bedetermined for the time period, and the activity costs and/or ratesadjusted (e.g., increased) to include the dedicated resource costs forthe time period. For example, suppose the customer provisions fiftyvirtual computers systems 214 (FIG. 2) and specifies that each of thefifty are to be implemented with dedicated implementation resources. Inresponse, the resource allocation component 1008 dedicates four VCSservers 222 to the customer, each capable of implementing sixteen of thevirtual computer systems requested by the customer, thus removing acapability of implementing sixty four such virtual computer systems fromthe general implementation resource pool 702 (FIG. 7). Further supposethe cost plan(s) 1022 for the customer's account specifies base rates of$0.10 per active dedicated resource hour and $0.05 per inactivededicated resource hour. Over a thousand hour time period, the customerincurs 50×1000×$0.10=$5000 in active costs and 14×1000×$0.05=$700 indedicated resource costs, for a total of $5700 in costs. As analternative to presenting separate items for active costs and dedicatedresource costs, the cost statement for the time period may present asingle item corresponding to 50×1000×$0.114=$5700. That is, in thisexample, the customer's base rate per active resource hour is adjustedto cover total costs for the time period.

As another example, activity costs and/or rates may be adjusted on a pervirtual resource type and/or dedicated implementation resource typebasis. Activity costs and dedicated resource costs corresponding to aparticular type of virtual resource and/or dedicated implementationresource may be determined for the cost statement time period, and theactivity costs and/or rates adjusted to include the dedicated resourcecosts for the time period. For example, suppose that customers of thevirtual resource provider 206 (FIG. 2) collectively provision virtualresources corresponding to one million resource allocation units of aparticular type implemented with dedicated implementation resources inone or more dedicated pools 704, 706 (FIG. 7), and that, as part of thededication process, a further twenty thousand resource allocation unitsof that type were transitioned from the general implementation resourcepool 702 but remain inactive. Collectively, the virtual resourceprovider has an inactive dedicated resource allocation unit to activededicated resource allocation unit ratio (an “inactive dedicated ratio”denoted herein with the letter a) of 2% in this example. The inactivededicated ratio may also be determined from implementation resourceutilization amounts. Further suppose that the customers are allocatedcosts in accordance with a common cost plan again specifying a base rateof $0.10 per active dedicated resource hour (c_(active)) and $0.05 perinactive dedicated resource hour (c_(inactive)). The adjusted rate peractive dedicate resource hour (c_(active)) may be determined with theformula:

c′ _(active) =c _(active) +α c _(inactive).

That is, $0.10+2%×$0.05=$0.101, in this example.

Adjustments to account for dedicated resource costs may be determinedwith respect to a set of time periods, for example, including one ormore previous time periods and/or forecast resource allocation unitutilization in one or more future time periods. When different customersand/or provisioned resource allocation units are associated withdifferent cost plans, adjustments corresponding to the cost plans may bedifferent, for example, weighted based at least in part on cost planattributes such as cost plan type, associated customer type,profitability and/or differing base rates. In addition, activity costsand/or dedicated resource costs may be modified based at least in parton resource allocation units corresponding to implementation resourcesin reserved implementation resource pools such as the reservedimplementation resource pool 708 of FIG. 7. For example, active and/orinactive dedicated resource hours may be offset by reserved resourcehours, and costs associated with reserved resource hours may bepresented as a separate item on the responsible customer's coststatement. Further in addition, the cost plan(s) 1022 may specify tieredcosts and/or resource hour rates, for example, a set of decreasing costscorresponding to a set of increasing virtual resource quantitythresholds (e.g., with respect to virtual resources of a same type)and/or a set of decreasing rates corresponding to a set of increasingresource hour quantity thresholds. Still further in addition, the costplan(s) 1022 may specify costs associated with dedicated implementationresource pool 704-706 transitioning procedures.

The description now turns to example steps and/or procedures that may beperformed in accordance with at least one embodiment. FIG. 11 depictsexample steps for virtual provisioning in accordance with at least oneembodiment. At step 1102, implementation resource pools may bemaintained. For example, the resource allocation component 1008 (FIG.10) may maintain the general implementation resource pool 702 of FIG. 7(step 1104) and one or more dedicated implementation resource pools704-706 (step 1106). In at least one embodiment, maintenance of thededicated implementation resource pools 704-706 includes migration ofvirtual resources among the implementation resources in a particulardedicated pool so as to maximize virtual resource density with respectto implementation resources and/or to progress towards virtual resourcespread targets. However, dedicated implementation resource pools 704-706need not be established and/or maintained prior to provisioning at leastone virtual resource requiring dedicated implementation resources.

At step 1108, a virtual resource provisioning request may be received.For example, the provisioning interface 1004 (FIG. 10) may receive thevirtual resource provisioning request from one of the clients 202-204(FIG. 2) responsive to virtual resource provider 206 customerinteraction with the client. The virtual resource provisioning requestmay incorporate and/or reference a set of implementation resourceconstraints. For example, the set of implementation resource constraintsmay include constraints with respect to virtual resource density and/orvirtual resource spread, constraints with respect to dedicatedimplementation resources including that particular types ofimplementation resources are to be dedicated to virtual resources of thecustomer, and constraints with respect to implementation resourceattributes such as geographic location. At step 1110, a set ofimplementation resources required to implement the virtual resourcerequested at step 1108 may be determined. For example, the resourceallocation component 1008 may determine the required set ofimplementation resources based at least in part on one or moreparameters and/or attributes of the virtual resource provisioningrequest including the set of implementation resource constraints.

At step 1112, it may be determined whether the set of requiredimplementation resources determined at step 1110 are available. Forexample, the resource allocation component 1008 may track availabilityof implementation resources in the general pool 702 (FIG. 7) and thededicated pools 704, 706, and compare the set of required implementationresources with an available implementation resource inventory. If eachof the set of required implementation resources is available, aprocedure incorporating step 1112 may progress to step 1114. Otherwise,the procedure may progress to step 1116 where the availability problemmay be reported to the party that sent the provisioning request receivedat step 1108.

At step 1114, the set of required implementation resources may beallocated. For example, the resource allocation component 1008 (FIG. 10)may update the available implementation resource inventory andtransition implementation resources between pools 702-706 (FIG. 7) asnecessary. The resource allocation component 1008 may select from amongthe available implementation resource in accordance with the set ofimplementation resource constraints. For example, the resourceallocation component 1008 may select implementation resources withgeographical separations that satisfy a constraint specifying a targetgeographic separation. If the set of implementation resource constraintsincludes a constraint specifying and/or referencing a virtual resourcespread target, the resource allocation component 1008 may invoke thespread component 1016 select and/or reallocate implementation resourcesin accordance with the virtual resource spread target. When the set ofrequired implementation resources includes dedicated implementationresources, the spread component 1016 may furthermore attempt to maximizevirtual resource density while respecting implementation resourcevirtual resource capacities and/or the virtual resource spread target.

At step 1118, the requested virtual resource may be provisioned with theset of required implementation resources allocated at step 1114. Forexample, upon receiving notification from the resource allocationcomponent 1008 that the resource allocation of step 1114 was successful,the provisioning interface 1004 may create a suitable provisioningworkflow.

FIG. 12 depicts example steps for determining implementation resourceavailability in accordance with at least one embodiment. At step 1202, anext (e.g., a first) implementation resource to be checked foravailability may be selected from a set of required implementationresources. For example, the resource allocation component 1008 (FIG. 10)may select the next implementation resource from the set of requiredimplementation resources determined at step 1110 of FIG. 11. At step1204, it may be determined whether allocating the selectedimplementation resource would violate one or more resource allocationbusiness policies of the virtual resource provider 206 (FIG. 2). Forexample, the business policy enforcement component 1014 may evaluate theselected implementation resource allocation with respect to the set ofresource allocation business policies. In this example described withreference to FIG. 12, step 1204 is performed for each implementationresource in the set of required implementation resources. Alternatively,or in addition, the determination of step 1204 may be performed once forthe set of required implementation resources as a whole.

At step 1206, an attempt to allocate the implementation resourceselected at step 1202 (a “pre-allocation check”) may be made. Forexample, the resource allocation component 1008 (FIG. 10) may search theappropriate implementation resource pool 702-706 (FIG. 7) for anunallocated implementation resource of a type matching the selectedimplementation resource. The actual allocation may be delayed (e.g., tostep 1114 of FIG. 11) to ensure atomic allocation transactions (in thesense of database theory). Alternatively, partial allocations may beallowed and/or step 1206 may allocate the selected implementationresource if possible. At step 1208, it may be determined whether theallocation attempt of step 1206 was successful. If so, a procedureincorporating step 1208 may progress to step 1210. Otherwise, theprocedure may progress to one or more steps not shown in FIG. 12 (e.g.,step 1116 of FIG. 11) with a status 1212 indicating that at least oneimplementation resource is not available.

At step 1210, it may be determined whether there are more implementationresources to be checked in the set of required implementation resources.If so, the procedure may progress to step 1202. Otherwise, the proceduremay progress to one or more steps not shown in FIG. 12 (e.g., step 1114of FIG. 11) with a status 1214 indicating that each of the set ofrequired implementation resources is available.

FIG. 13 depicts example steps for allocating and/or attempting toallocate an implementation resource in accordance with at least oneembodiment. For example, the steps of FIG. 13 may be incorporated intostep 1112 or step 1114 of FIG. 11 and/or step 1206 of FIG. 12. At step1302, it may be determined whether an implementation resource to beallocated (the “allocation candidate”) is to be allocated from aparticular dedicated implementation resource pool. For example, the setof implementation resources determined at step 1110 of FIG. 11 mayinclude a set of implementation resources to be allocated from thededicated implementation resource pool 706 of FIG. 7, and the allocationcandidate may be one of those, and/or the resource allocation component1008 (FIG. 10) may identify one or more dedicated implementationresource constraints associated with the virtual resource provisioningrequest received at step 1108 of FIG. 11. If the allocation candidate isto be allocated from one of the dedicated implementation resource pools704-706, a procedure incorporating step 1302 may progress to step 1304.Otherwise, the procedure may progress to step 1306.

At step 1306, it may be determined whether an implementation resourcematching the allocation candidate is available (e.g., unallocated) inthe general implementation resource pool 702 (FIG. 7). For example, theresource allocation component 1008 (FIG. 10) may search the generalimplementation resource pool 702 for an unallocated implementationresource of a type that matches the allocation candidate. If theallocation candidate is available in the general implementation resourcepool 702, the procedure may progress to step 1308. Otherwise, theprocedure may progress to one or more steps not shown in FIG. 13, forexample, step 1116 of FIG. 11, with a status 1310 indicating that theallocation candidate could not be allocated. At step 1308, theallocation candidate may be allocated from the general implementationresource pool 702. For example, the resource allocation component 1008(FIG. 10) may record the general pool 702 implementation resource foundat step 1306 as allocated to an associated virtual resource (e.g., thevirtual resource associated with the provisioning request of step 1108of FIG. 11).

At step 1304, it may be determined whether an implementation resourcematching the allocation candidate is available in the particulardedicated implementation resource pool determined at step 1302 (the“candidate dedicated pool”). For example, the resource allocationcomponent 1008 (FIG. 10) may search the dedicated implementationresource pool 706 (FIG. 7) for an unallocated implementation resource ofa type that matches the allocation candidate. If the allocationcandidate is available in the candidate dedicated pool, the proceduremay progress to step 1312. Otherwise, the procedure may progress to step1314. At step 1312, the allocation candidate may be allocated from thecandidate dedicated pool. For example, the resource allocation component1008 may record the dedicated pool 706 implementation resource found atstep 1304 as allocated to the associated virtual resource.

If it is determined that the allocation candidate is not currentlyavailable in the candidate dedicated pool, at step 1314, it may furtherbe determined whether the candidate dedicated pool is expandable. Forexample, the dedicated resource pools component 1012 (FIG. 10) may checkthe associated cost plan(s) 1022 and/or request an evaluation of anexpansion of the candidate dedicated pool by the business policyenforcement component 1014. If the candidate dedicated pool isexpandable, the procedure may progress to step 1316, otherwise, theprocedure may progress to one or more steps not shown in FIG. 13, forexample, step 1116 of FIG. 11, with the status 1310 indicating that theallocation candidate could not be allocated.

At step 1316, it may be determined whether an implementation resourcematching the allocation candidate is available in the generalimplementation resource pool 702 (FIG. 7), for example, as describedabove for step 1306. If so, the procedure may progress to step 1318.Otherwise, the procedure may progress to one or more steps not shown inFIG. 13, for example, step 1116 of FIG. 11, with the status 1310indicating that the allocation candidate could not be allocated. At step1318, the implementation resource found at step 1316 may be transitionedto the candidate dedicated pool (and/or the transition may beinitiated). For example, the dedicated resource pools component 1012 mayrequest the pool transition component 1018 to transition the generalpool 702 implementation resource found at step 1316. The transitionedimplementation resource may be allocated as described above for step1312. When the steps of FIG. 13 are being performed as part of apre-allocation check, step 1308, step 1312 and step 1318 may be omittedand/or correspond to a “resource available” status.

It may be that the set of implementation resources determined at step1110 of FIG. 11 includes a set of implementation resources to beallocated from a dedicated implementation resource pool that does notyet exist. In this case the dedicated resource pools component 1012(FIG. 10) may establish a new dedicated implementation resource poolincluding the set of implementation resources to be allocated. FIG. 14depicts example steps for establishing a dedicated implementationresource pool in accordance with at least one embodiment. At step 1402,a request to establish a new dedicated implementation resource pool (a“new dedicated pool request”) may be received. For example, thededicated resource pools component 1012 may receive the requestresponsive to the virtual resource provisioning request of step 1108 ofFIG. 11.

At step 1404, the new dedicated pool request may be mapped to a set ofimplementation resources. The new dedicated pool request may explicitlyspecify a set of implementation resources to be added to the newdedicated implementation resource pool. Alternatively, or in addition,the new dedicated pool request may specify a set of virtual resources tobe implemented with dedicated implementation resources, and thededicated resource pools component 1012 (FIG. 10) may map the set ofvirtual resources to the set of implementation resources.

At step 1406, the new dedicated pool request may be evaluated withrespect to the set of resource allocation business policies, forexample, by the business policy enforcement component 1014 (FIG. 10). Atstep 1408, it may be determined whether the new dedicated pool requestis in accordance with the et of resource allocation business policiesbased at least in part on the evaluation of step 1406. If so, aprocedure incorporating step 1408 may progress to step 1410. Otherwise,the problem may be reported to the requester at step 1412.

At step 1410, it may be determined whether the set of implementationresources obtained at step 1404 is available in the generalimplementation resource pool 702 (FIG. 2), for example, as describedabove for step 1316 of FIG. 13. If so, the procedure may progress tostep 1414. Otherwise, the procedure may progress to step 1412 to reportthe problem to the requester. At step 1414, the new dedicatedimplementation resource pool may be added to the set of dedicatedimplementation resource pools 704-706 (FIG. 7), for example, by thededicated resource pools component 1012 (FIG. 10). At step 1416, the setof implementation resources identified as available in the generalimplementation resource pool 702 may be transitioned to the newdedicated implementation resource pool.

As described above with reference to FIG. 10, dedicated implementationresource pools may be established with an inactive dedicatedimplementation resource buffer. FIG. 15 depicts example steps formaintaining an inactive dedicated implementation resource buffer inaccordance with at least one embodiment. At step 1502, dedicated poolutilization data may be collected. For example, the pool utilizationcomponent 1020 may monitor levels of activity and/or inactivity in thededicated implementation resource pool 706 (FIG. 7). At step 1504,future dedicated pool utilization may be forecast based at least in parton the dedicated pool utilization data collected at step 1502, forexample, by the pool utilization component 1020.

At step 1506, it may be determined whether a current inactive dedicatedimplementation resource buffer is sufficient. For example, the poolutilization component 1020 may compare a forecast level of inactivity inthe dedicated implementation resource pool 706 (FIG. 7) to a targetlevel of inactivity for the dedicated implementation resource pool 706and/or a minimum inactivity buffer threshold. If the current buffer issufficient (e.g., the forecast level of inactivity meets or exceeds theminimum inactivity buffer threshold), a procedure incorporating step1506 may progress to step 1508. Otherwise, the procedure may progress tostep 1510. At step 1510, one or more implementation resources may betransitioned from the general implementation resource pool 702 (e.g.,from the reserved portion 708 of the general implementation resourcepool 702) to the dedicated implementation resource pool 706. Forexample, the pool utilization component 1020 may request the pooltransition component 1018 to transition sufficient implementationresources to the dedicated implementation resource pool 706 so that theforecast buffer is sufficient.

At step 1508, it may be determined whether a current inactive dedicatedimplementation resource buffer is excessive. For example, the poolutilization component 1020 may compare the forecast level of inactivityin the dedicated implementation resource pool 706 (FIG. 7) to the targetlevel of inactivity for the dedicated implementation resource pool 706and/or a maximum inactivity buffer threshold. If the current buffer isexcessive (e.g., the forecast level of inactivity exceeds the maximuminactivity buffer threshold), the procedure may progress to step 1512.Otherwise, the procedure may progress to step 1502 to collect more data.At step 1512, one or more implementation resources may be transitionedfrom the dedicated implementation resource pool 706 to the generalimplementation resource pool 702 (e.g., to the reserved portion 708 ofthe general implementation resource pool 702). For example, the poolutilization component 1020 may request the pool transition component1018 to transition sufficient implementation resources to the generalimplementation resource pool 706 so that the forecast level ofinactivity corresponds to the target level of inactivity for thededicated implementation resource pool 706.

Costs associated with maintaining provisioned virtual resources may bepresented to responsible customers of the virtual resource provider 206(FIG. 2) according to cost presentation schedules specified by anadministrator of the virtual resource provider 206 and/or the customers.FIG. 16 depicts example steps for tracking costs in accordance with atleast one embodiment. At step 1602, a set of virtual resources may bemaintained during a time period. For example, the virtual resourceprovider 206 may maintain a set of the virtual resources 212 for aparticular customer during the time period.

At step 1604, one or more active implementation resource utilizationamounts corresponding to the set of virtual resources maintained at step1602 may be determined. For example, the accounting component 1010 ofFIG. 10 may determine a number of active resource hours associated witheach of the set of virtual resources during the time period. At step1606, one or more inactive implementation resource utilization amountscorresponding to the set of virtual resources maintained at step 1602may be determined. For example, the accounting component 1010 maydetermine a number of inactive resource hours associated with each ofthe set of virtual resources during the time period.

At step 1608, it may be determined whether the set of virtual resourcesare associated with a set of reserved virtual resources. For example,the customer may provision the set of reserved virtual resources withthe provisioning interface 1004 (FIG. 10) and/or associate the set ofreserved virtual resources with the previously provisioned set ofvirtual resources maintained at step 1602. If there are reserved virtualresources associated with the set of virtual resources for which costsare being determined, a procedure incorporating step 1608 may progressto step 1610. Otherwise the procedure may progress to step 1612. At step1610, the set of reserved virtual resources may be mapped to one or morereserved implementation resource utilization amounts. For example, theaccounting component 1010 of FIG. 10 may determine a number of reservedresource hours associated with each of the set of reserved virtualresources.

At step 1612, one or more costs may be determined in accordance with oneor more cost plans associated with the customer and/or the set ofvirtual resources. For example, the accounting component 1010 (FIG. 10)may determine the cost(s) based at least in part on the cost plan(s)1022 and/or the active, inactive and/or reserved implementation resourceutilization amounts determined at step 1604, step 1606 and/or step 1610,respectively. At step 1614, the cost(s) determined at step 1612 may bepresented to the customer. For example, the accounting component 1010may generate one or more cost presentations accessible with the accountUI 1024.

As described above with reference to FIG. 10, the control plane 1002 maybe facilitated by one or more workflows maintained by the workflowcomponent 1006. FIG. 17 depicts example steps for workflow in accordancewith at least one embodiment. At step 1702, a request may be received byan interface of the control plane 1002 (FIG. 10). For example, theprovisioning interface 1004 and/or the account UI 1024 of the controlplane 1002 may receive the request from a customer and/or administratorof the virtual resource provider 206. The request may be a call to aprogrammatic interface such as an application programming interface(API) or a Web services interface, e.g., utilizing a Simple ObjectAccess Protocol (SOAP) call. Alternatively, or in addition, the requestmay be received through a graphical user interface (GUI) such as aWeb-based GUI. At step 1704, the request may be analyzed to determineone or more actions required to successfully complete the request. Forexample, the provisioning interface 208 may analyze the request, anddetermine a set of actions required to provision a set of virtualresources. When an interface element receiving the request correspondsto a specific action to be performed, the interface may extractinformation from the request to be utilized in determining aspectsand/or parameters of the action to be performed.

At step 1706, a request may be sent to create a workflow based at leastin part on the one or more actions determined at step 1704. For example,provisioning interface 1004

(FIG. 10) may send the request to the workflow component 1006. Therequest to create the workflow may include the action(s), actionmetadata such as type of action, and/or action parameters. In at leastone embodiment, the control plane 1002 and/or the workflow component1006 maintains a job queue for such requests, and workflows are createdresponsive to new additions to the job queue. At step 1708, a workflowand one or more component tasks may be created. For example, theworkflow component 1006 may analyze the request of step 1706 todetermine the appropriate workflow and component tasks to create.

At step 1710, execution of the component task(s) may be guided inaccordance with the workflow. For example, the workflow component 1006(FIG. 10) may activate elements of interfaces of the implementationresources 220 (FIG. 2) to provision the set of virtual resources.Alternatively, or in addition, the workflow component 1006 may managebids for execution of the component task(s) by components of the virtualresource provider 206. At step 1712, it may be determined whether theworkflow has finished. For example, the workflow component 1006 maydetermine whether a final task in a sequence of tasks managed by theworkflow has completed. If so, a procedure incorporating step 1712 mayprogress to step 1714. Otherwise the procedure may return to step 1710for a next task and/or task sequence. Workflows may guide multiple tasksequences executing in parallel. In this case, it may be that theworkflow is not finished until each of the multiple task sequencescompletes and/or an explicit workflow finished flag is set by one of thecomponent tasks. At step 1714, the sender of the request of step 1702may be informed of the result(s) of the action(s).

The various embodiments described herein may be implemented in a widevariety of operating environments, which in some cases may include oneor more user computers, computing devices, or processing devices whichmay be utilized to operate any of a number of applications. User orclient devices may include any of a number of general purpose personalcomputers, such as desktop or laptop computers running a standardoperating system, as well as cellular, wireless, and handheld devicesrunning mobile software and capable of supporting a number of networkingand messaging protocols. Such a system also may include a number ofworkstations running any of a variety of commercially-availableoperating systems and other known applications for purposes such asdevelopment and database management. These devices also may includeother electronic devices, such as dummy terminals, thin-clients, gamingsystems, and other devices capable of communicating via a network.

Most embodiments utilize at least one network that would be familiar tothose skilled in the art for supporting communications using any of avariety of commercially-available protocols, such as TCP/IP, OSI, FTP,UPnP, NFS, CIFS, and AppleTalk. Such a network may include, for example,a local area network, a wide-area network, a virtual private network,the Internet, an intranet, an extranet, a public switched telephonenetwork, an infrared network, a wireless network, and any combinationthereof. The network may, furthermore, incorporate any suitable networktopology. Examples of suitable network topologies include, but are notlimited to, simple point-to-point, star topology, self organizingpeer-to-peer topologies, and combinations thereof.

In embodiments utilizing a Web server, the Web server may run any of avariety of server or mid-tier applications, including HTTP servers, FTPservers, CGI servers, data servers, Java servers, and businessapplication servers. The server(s) also may be capable of executingprograms or scripts in response requests from user devices, such as byexecuting one or more Web applications that may be implemented as one ormore scripts or programs written in any programming language, such asJava®, C, C# or C++, or any scripting language, such as Perl, Python, orTCL, as well as combinations thereof. The server(s) may also includedatabase servers, including without limitation those commerciallyavailable from Oracle°, Microsoft°, Sybase°, and IBM®.

The environment may include a variety of data stores and other memoryand storage media as discussed above. These may reside in a variety oflocations, such as on a storage medium local to (and/or resident in) oneor more of the computers or remote from any or all of the computersacross the network. In a particular set of embodiments, the informationmay reside in a storage-area network (“SAN”) familiar to those skilledin the art. Similarly, any necessary files for performing the functionsattributed to the computers, servers, or other network devices may bestored locally and/or remotely, as appropriate. Where a system includescomputerized devices, each such device may include hardware elementsthat may be electrically coupled via a bus, the elements including, forexample, at least one central processing unit (CPU), at least one inputdevice (e.g., a mouse, keyboard, controller, touch screen, or keypad),and at least one output device (e.g., a display device, printer, orspeaker). Such a system may also include one or more storage devices,such as disk drives, optical storage devices, and solid-state storagedevices such as random access memory (“RAM”) or read-only memory(“ROM”), as well as removable media devices, memory cards, flash cards,etc.

Such devices also may include a computer-readable storage media reader,a communications device (e.g., a modem, a network card (wireless orwired), an infrared communication device, etc.), and working memory asdescribed above. The computer-readable storage media reader may beconnected with, or configured to receive, a computer-readable storagemedium, representing remote, local, fixed, and/or removable storagedevices as well as storage media for temporarily and/or more permanentlycontaining, storing, transmitting, and retrieving computer-readableinformation. The system and various devices also typically will includea number of software applications, modules including program modules,services, or other elements located within at least one working memorydevice, including an operating system and application programs, such asa client application or Web browser. It should be appreciated thatalternate embodiments may have numerous variations from that describedabove. For example, customized hardware might also be utilized and/orparticular elements might be implemented in hardware, software(including portable software, such as applets), or both. Further,connection to other computing devices such as network input/outputdevices may be employed.

Storage media and computer readable media for containing code, orportions of code, may include any appropriate media known or used in theart, including storage media and communication media, such as but notlimited to volatile and non-volatile, removable and non-removable mediaimplemented in any method or technology for storage and/or transmissionof information such as computer readable instructions, data structures,program modules, or other data, including RAM, ROM, EEPROM, flash memoryor other memory technology, CD-ROM, digital versatile disk (DVD) orother optical storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, or any other medium which maybe utilized to store the desired information and which may be accessedby the a system device. Program modules, program components and/orprogrammatic objects may include computer-readable and/orcomputer-executable instructions of and/or corresponding to any suitablecomputer programming language. In at least one embodiment, eachcomputer-readable medium may be tangible. In at least one embodiment,each computer-readable medium may be non-transitory in time. Based onthe disclosure and teachings provided herein, a person of ordinary skillin the art will appreciate other ways and/or methods to implement thevarious embodiments.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that various modifications and changes may be made thereuntowithout departing from the broader spirit and scope of the invention asset forth in the claims.

The use of the terms “a” and “an” and “the” and similar referents in thecontext of describing embodiments (especially in the context of thefollowing claims) are to be construed to cover both the singular and theplural, unless otherwise indicated herein or clearly contradicted bycontext. The terms “comprising,” “having,” “including,” and “containing”are to be construed as open-ended terms (i.e., meaning “including, butnot limited to,”) unless otherwise noted. The term “connected” is to beconstrued as partly or wholly contained within, attached to, or joinedtogether, even if there is something intervening. Recitation of rangesof values herein are merely intended to serve as a shorthand method ofreferring individually to each separate value falling within the range,unless otherwise indicated herein, and each separate value isincorporated into the specification as if it were individually recitedherein. All methods described herein can be performed in any suitableorder unless otherwise indicated herein or otherwise clearlycontradicted by context. The use of any and all examples, or exemplarylanguage (e.g., “such as”) provided herein, is intended merely to betterilluminate embodiments and does not pose a limitation on the scopeunless otherwise claimed. No language in the specification should beconstrued as indicating any non-claimed element as essential to thepractice of at least one embodiment.

Preferred embodiments are described herein, including the best modeknown to the inventors. Variations of those preferred embodiments maybecome apparent to those of ordinary skill in the art upon reading theforegoing description. The inventors expect skilled artisans to employsuch variations as appropriate, and the inventors intend for embodimentsto be constructed otherwise than as specifically described herein.Accordingly, suitable embodiments include all modifications andequivalents of the subject matter recited in the claims appended heretoas permitted by applicable law. Moreover, any combination of theabove-described elements in all possible variations thereof iscontemplated as being incorporated into some suitable embodiment unlessotherwise indicated herein or otherwise clearly contradicted by context.

All references, including publications, patent applications, andpatents, cited herein are hereby incorporated by reference to the sameextent as if each reference were individually and specifically indicatedto be incorporated by reference and were set forth in its entiretyherein.

1-27. (canceled)
 28. A computer-implemented method, comprising:receiving, at a control plane of a virtual machine provider, a requestto create a dedicated hardware resource for hosting a virtual resourceof a customer; creating the dedicated hardware resource by dedicating,from a general hardware resource pool, a general hardware resource tohosting the virtual resource of the customer, wherein creating thededicated hardware resource establishes one or more information barriersbetween the dedicated hardware resource and other resources of othercustomers; and after creating the dedicated hardware resource, launchingthe virtual resource using the dedicated hardware resource.
 29. Thecomputer-implemented method of claim 28, wherein the request specifiesthe general hardware resource, and wherein the method further comprisesselecting the general hardware resource from the general hardwareresource pool prior to creating the dedicated hardware resource bydedicating the general hardware resource to hosting the virtual resourceof the customer.
 30. The computer-implemented method of claim 28,wherein the dedicated hardware resource is located within a mobile datacenter.
 31. The computer-implemented method of claim 30, wherein themobile data center comprises a self-contained datacenter unit comprisingat least server racks and network racks.
 32. The computer-implementedmethod of claim 28, wherein creating the dedicated hardware resource bydedicating the general hardware resource to hosting the virtual resourcecomprises transitioning the general hardware resource from the generalhardware resource pool to a dedicated hardware resource pool inaccordance with a set of transition procedures, the dedicated hardwareresource pool being dedicated to the customer.
 33. Thecomputer-implemented method of claim 28, wherein the control planecomprises a provisioning interface and a resource allocation component,wherein receiving the request comprises receiving the request at theprovisioning interface of the control plane, and wherein creating thededicated hardware resource comprises creating the dedicated hardwareresource using the resource allocation component.
 34. Thecomputer-implemented method of claim 28, wherein the one or moreinformation barriers physically and logically isolate the dedicatedhardware resource from the other resources of the other customers. 35.One or more non-transitory computer-readable media comprisingcomputer-executable instructions that, when executed by one or moreprocessors, cause one or more computer systems to perform operations,comprising: receiving, at a control plane of a virtual machine provider,a request to create a dedicated hardware resource for hosting a virtualresource of a customer; creating the dedicated hardware resource bydedicating, from a general hardware resource pool, a general hardwareresource to hosting the virtual resource of the customer, whereincreating the dedicated hardware resource establishes one or moreinformation barriers between the dedicated hardware resource and otherresources of other customers; and after creating the dedicated hardwareresource, launching the virtual resource using the dedicated hardwareresource.
 36. The one or more non-transitory computer-readable media ofclaim 35, wherein the request specifies the general hardware resource,and wherein the method further comprises selecting the general hardwareresource from the general hardware resource pool prior to creating thededicated hardware resource by dedicating the general hardware resourceto hosting the virtual resource of the customer.
 37. The one or morenon-transitory computer-readable media of claim 35, wherein thededicated hardware resource is located within a mobile data center. 38.The one or more non-transitory computer-readable media of claim 37,wherein the mobile data center comprises a self-contained datacenterunit comprising at least server racks and network racks.
 39. The one ormore non-transitory computer-readable media of claim 35, whereincreating the dedicated hardware resource by dedicating the generalhardware resource to hosting the virtual resource comprisestransitioning the general hardware resource from the general hardwareresource pool to a dedicated hardware resource pool in accordance with aset of transition procedures, the dedicated hardware resource pool beingdedicated to the customer.
 40. The one or more non-transitorycomputer-readable media of claim 35, wherein the control plane comprisesa provisioning interface and a resource allocation component, whereinreceiving the request comprises receiving the request at theprovisioning interface of the control plane, and wherein creating thededicated hardware resource comprises creating the dedicated hardwareresource using the resource allocation component.
 41. The one or morenon-transitory computer-readable media of claim 35, wherein the one ormore information barriers physically and logically isolate the dedicatedhardware resource from the other resources of the other customers.
 42. Asystem, comprising: one or more processors; and one or morenon-transitory computer readable storage media comprising instructionsthat, upon execution by the one or more processors, cause the system toat least: receive, at a control plane of a virtual machine provider, arequest to create a dedicated hardware resource for hosting a virtualresource of a customer; create the dedicated hardware resource bydedicating, from a general hardware resource pool, a general hardwareresource to hosting the virtual resource of the customer, whereincreating the dedicated hardware resource establishes one or moreinformation barriers between the dedicated hardware resource and otherresources of other customers; and after creating the dedicated hardwareresource, launch the virtual resource using the dedicated hardwareresource.
 43. The system of claim 42, wherein the request specifies thegeneral hardware resource, and wherein the method further comprisesselecting the general hardware resource from the general hardwareresource pool prior to creating the dedicated hardware resource bydedicating the general hardware resource to hosting the virtual resourceof the customer.
 44. The system of claim 42, wherein the dedicatedhardware resource is located within a mobile data center.
 45. The systemof claim 44, wherein the mobile data center comprises a self-containeddatacenter unit comprising at least server racks and network racks. 46.The system of claim 42, wherein creating the dedicated hardware resourceby dedicating the general hardware resource to hosting the virtualresource comprises transitioning the general hardware resource from thegeneral hardware resource pool to a dedicated hardware resource pool inaccordance with a set of transition procedures, the dedicated hardwareresource pool being dedicated to the customer.
 47. The system of claim42, wherein the control plane comprises a provisioning interface and aresource allocation component, wherein receiving the request comprisesreceiving the request at the provisioning interface of the controlplane, and wherein creating the dedicated hardware resource comprisescreating the dedicated hardware resource using the resource allocationcomponent.